Bayes Theorem for Improper Priors

Improper priors are used frequently, but often formally and without reference to a sound theoretical basis. A consequence is the occurrence of seemingly paradoxical results. The most famous example is perhaps given by the marginalization paradoxes presented by Stone and Dawid (1972). It is demonstrated here that the seemingly paradoxical results are removed by a more careful formulation of the theory. The present paper demonstrates more generally that Kolmogorov’s (1933) formulation of probability theory admits a minimal generalization which includes improper priors and a general Bayes theorem. It is interesting that the resulting theory is closely related to the theory formulated by Renyi (1970), but the initial axioms and the motivation differ. The resulting theory includes improper priors, explains the marginalization paradoxes, and gives conditions which ensure propriety of the resulting posteriors. These results are relevant for the current usage of improper priors.